Machine Learning
FS 2019
Antonio Rosolia
10.06.2019

Machine Learning End to End Project - Flower Recognition with CNN

Introduction

The flower dataset contains 4242 images of flowers. The data collection is based on the data flicr, google images, yandex images. The pictures are divided into five classes: chamomile, tulip, rose, sunflower, dandelion. For each class there are about 800 photos. Photos are not high resolution, about 320x240 pixels. Photos are not reduced to a single size, they have different proportions!

This dataset can be found on https://www.kaggle.com/alxmamaev/flowers-recognition

Objectives

The aim of this project is to classify a photo of a plant to the correct plant species. The image classification is done using Deep Learning.

Approach

In this project Keras is used to build a convolutional neural network for image classification.

The following layers will be added to the CNN:
-Convolutional Layer
-Pooling Layer
-Flattening Layer
-Neural Network
Arichitecture of a CNN -  Source: https://www.mathworks.com/videos/introduction-to-deep-learning-what-are-convolutional-neural-networks--1489512765771.html

Note

This Jupyter notebook was run on a Google Cloud VM with GPU.

Explorative Data Analysis

In [1]:
#importing libraries
import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import shutil
import warnings
import tensorflow as tf
import imageio
from pathlib import Path
from sklearn.model_selection import train_test_split
from keras.preprocessing.image import ImageDataGenerator
from sklearn.preprocessing import LabelEncoder
warnings.simplefilter('ignore')


%matplotlib inline
Using TensorFlow backend.

All pictures are going to be packed together with the category in a Dataframe.

In [214]:
# Path of the Images
input_path = './data/flowers'

# Get all the species
flower_types = os.listdir(input_path)
print("Number of flower species: ", len(flower_types))
print("Flower species: ", flower_types)
Number of flower species:  5
Flower species:  ['sunflower', 'rose', 'dandelion', 'tulip', 'daisy']
In [215]:
flowers = []

for species in flower_types:
    # Get all the file names
    all_flowers = os.listdir(input_path +'/'+ species)
    # Add them to the list
    for flower in all_flowers:
        flowers.append((species, str(input_path +'/'+species) + '/' + flower))

# Build a dataframe        
df_flower = pd.DataFrame(data=flowers, columns=['category', 'image'], index=None)
df_flower.head()
Out[215]:
category image
0 sunflower ./data/flowers/sunflower/5004121118_e9393e60d0...
1 sunflower ./data/flowers/sunflower/7721658400_0dec46d225...
2 sunflower ./data/flowers/sunflower/2883115609_5a69357b5d...
3 sunflower ./data/flowers/sunflower/3154932076_eff5c38231...
4 sunflower ./data/flowers/sunflower/22405882322_d4561f846...
In [4]:
df_flower['category'].value_counts()
Out[4]:
dandelion    1052
tulip         984
rose          784
daisy         769
sunflower     734
Name: category, dtype: int64
In [5]:
df_flower['category'].value_counts().plot(kind="bar")
plt.ylabel('count')
Out[5]:
Text(0, 0.5, 'count')

As we can see in the above barplot, all plants have about the same number of pictures.

In addition, all images are checked to ensure they are in jpg format.

In [216]:
for index, cell in df_flower['image'].iteritems():
    if(cell.split(".")[2]!="jpg"):
        print("Cell "+ str(index) + " contains a file which is not in jpg format. File: " + cell)
Cell 2082 contains a file which is not in jpg format. File: ./data/flowers/dandelion/flickr.py
Cell 2333 contains a file which is not in jpg format. File: ./data/flowers/dandelion/run_me.py
Cell 2554 contains a file which is not in jpg format. File: ./data/flowers/dandelion/flickr.pyc

There are some Python files (not sure why), which will be deleted. Also the dataframe is going to be recreated, since these python files are in it.

In [217]:
if os.path.exists("./flowers/dandelion/flickr.py"):
    os.remove("./flowers/dandelion/flickr.py")
if os.path.exists("./flowers/dandelion/run_me.py"):
    os.remove("./flowers/dandelion/run_me.py")
if os.path.exists("./flowers/dandelion/flickr.pyc"):
    os.remove("./flowers/dandelion/flickr.pyc")

# Get all the species
flower_types = os.listdir(input_path)

flowers = []

for species in flower_types:
    # Get all the file names
    all_flowers = os.listdir(input_path +'/'+ species)
    # Add them to the list
    for flower in all_flowers:
        flowers.append((species, str(input_path +'/'+species) + '/' + flower))

# Build a dataframe        
df_flower = pd.DataFrame(data=flowers, columns=['category', 'image'], index=None)

In addition, the dimensions of the images will be checked.

In [4]:
height=[]
width=[]
colors=[]

for img in df_flower['image']:
    height.append(imageio.imread(img).shape[0])
    width.append(imageio.imread(img).shape[1])
    colors.append(imageio.imread(img).shape[2])
print("Minimum height: ", min(height)," Minimum width: ",min(width)," Minimum Colors: ", min(colors))
Minimum height:  80  Minimum width:  134  Minimum Colors:  3
In [5]:
plt.hist(width, bins=100)
plt.title("Histogram from width")
plt.xlabel("width")
plt.ylabel("count")
plt.show()
In [6]:
plt.hist(height, bins=100)
plt.title("Histogram from height")
plt.xlabel("width")
plt.ylabel("count")
plt.show()

These histograms are important to determine the input shape from the images.

Finally let's have a look at some flowers. The following images are randomly selected. Each time the cell is executed, different images appear.

In [5]:
#creating random_sample array of imagepath which contains 5 images of each species
random_samples=[]
for species in flower_types:
    for sample in df_flower['image'].where(df_flower['category']==species).dropna().sample(5).values:
        random_samples.append(sample)

#ploting all the random samples
counter=0
fig,ax=plt.subplots(5,5)
fig.set_size_inches(15,15)
for i in range(5):
    for j in range (5):
        img= mpimg.imread(random_samples[counter],1)
        ax[i,j].imshow(img,cmap=None)
        ax[i,j].set_title('Flower: '+random_samples[counter].split("/")[2])
        counter+=1
        
        
plt.tight_layout()

Lovely. The images are going to be split in test and train sets. The size of the training set is 90%.

In [218]:
#Creating Train / Test folders
train_dir = '.data/train/'
test_dir = '.data/test/'

if not os.path.exists(train_dir):
    os.makedirs(train_dir)
if not os.path.exists(test_dir):
    os.makedirs(test_dir)
for species in flower_types:
    if not os.path.exists(train_dir+species):
        os.makedirs(train_dir+species)
        os.makedirs(test_dir+species)
        all_species_images = df_flower['image'].where(df_flower['category']==species).dropna()
        y = df_flower['category'].where(df_flower['category']==species).dropna()
        X = df_flower['image'].where(df_flower['category']==species).dropna()
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)
        for img in X_train:
            shutil.copy(img, ".data/train/"+species)
        for img in X_test:
            shutil.copy(img, ".data/test/"+species)

CNN Model

First a simple model is created and trained. Then a more complicated model is trained and compared to the simple model.

Since it is much faster to train and test a model on a GPU than on a CPU, the Tensorflow backend for Keras is going to be running on a GPU.
According to the Keras Documentation:

if you are running on the TensorFlow or CNTK backends, your code will automatically run on GPU if any available GPU is detected.

So the first step is to ensure that a GPU is available.

In [11]:
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 16365056253665686170
, name: "/device:XLA_GPU:0"
device_type: "XLA_GPU"
memory_limit: 17179869184
locality {
}
incarnation: 7672343860173145506
physical_device_desc: "device: XLA_GPU device"
, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 8649577177623917736
physical_device_desc: "device: XLA_CPU device"
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 11276946637
locality {
  bus_id: 1
  links {
  }
}
incarnation: 18263407997947442078
physical_device_desc: "device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7"
]

Since a GPU is available Keras should be using TensorFlow on it.

In [6]:
import keras

# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dropout
from keras.layers import Dense
from keras.models import Model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
In [6]:
# Initialising the CNN
classifier = Sequential()

Step 1 - Convolution Layer

[1]The first step of a CNN is the Convolution layer.

Convoluting a 5x5x1 image with a 3x3x1 kernel to get a 3x3x1 convolved feature

The objective of the convolution operation is to extract context and high-level features such as edges, from the input image. Also it helps to eliminate unnecessary informations and reduce dimensionality. The result of this layer are the Feature maps.

In [34]:
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
#32 feature detectors with the size of 3x3 will be applied on the input RGB images with size of 64x64

Step 2 - Max Pooling Layer

[2]This layer is quite similar to the Convolution Layer. It stores the largest number from a feature map to a pooled feature map.

By pooling these features it stores highly important parts from the feature maps. Furthermore the size will be decreased and this step also helps to reduce overfitting.

In [35]:
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
#a 2x2 pooling map will be applied to the feature maps

Step 3 Flattening Layer

[3]The features map will be flattened into one single vector.

No spatial information will be lost, since the pooled the information is from the pooled feature maps, which has the context in it. The final numbers in the vectors, are the maximum numbers, which the feature detector had extracted.

In [36]:
# Step 3 - Flattening
classifier.add(Flatten())

Step 4 - Full Connection

To classify the images, an artificial neural network will be used. It takes the flattening layer as input and the output will be one of the five categories. In the neural network the two following activation functions will be used:

ReLU (Rectified Linear Unit) Activation Function

[4]ReLU is the most commonly used activation function in neural networks, especially in CNNs.

  • It’s cheap to compute as there is no complicated math. The model can therefore take less time to train or run.
  • It converges faster. Linearity means that the slope doesn’t plateau, or “saturate,” when x gets large. It doesn’t have the vanishing gradient problem suffered by other activation functions like sigmoid or tanh.
  • It’s sparsely activated. Since ReLU is zero for all negative inputs, it’s likely for any given unit to not activate at all. This is often desirable.

Softmax Activation Function

[5]The Softmax function is mainly used for mulitclass categorical classification.

In [37]:
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 5, activation = 'softmax'))
In [38]:
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
In [39]:
classifier.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 62, 62, 32)        896       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 31, 31, 32)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 30752)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               3936384   
_________________________________________________________________
dense_2 (Dense)              (None, 5)                 645       
=================================================================
Total params: 3,937,925
Trainable params: 3,937,925
Non-trainable params: 0
_________________________________________________________________

Keras has ImageDataGenerator class which allows the users to perform image augmentation on the fly in a very easy way, so that the model would never see twice the exact same picture. This helps prevent overfitting and helps the model generalize better.

In [219]:
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)
In [220]:
training_set = train_datagen.flow_from_directory('.data/train',
                                                 target_size = (64, 64),
                                                 batch_size = 32,)

test_set = test_datagen.flow_from_directory('.data/test',
                                            target_size = (64, 64),
                                            batch_size = 32,)
Found 3888 images belonging to 5 classes.
Found 435 images belonging to 5 classes.
In [61]:
classifier.fit_generator(training_set,
                         steps_per_epoch = 3888,
                         epochs = 10,
                         validation_data = test_set,
                         validation_steps = 435)
Epoch 1/10
3888/3888 [==============================] - 793s 204ms/step - loss: 0.5355 - acc: 0.8001 - val_loss: 1.3561 - val_acc: 0.6685
Epoch 2/10
3888/3888 [==============================] - 790s 203ms/step - loss: 0.1460 - acc: 0.9510 - val_loss: 1.8817 - val_acc: 0.7013
Epoch 3/10
3888/3888 [==============================] - 808s 208ms/step - loss: 0.0775 - acc: 0.9752 - val_loss: 2.0277 - val_acc: 0.6846
Epoch 4/10
3888/3888 [==============================] - 803s 207ms/step - loss: 0.0584 - acc: 0.9817 - val_loss: 2.1429 - val_acc: 0.7110
Epoch 5/10
3888/3888 [==============================] - 792s 204ms/step - loss: 0.0488 - acc: 0.9851 - val_loss: 2.3503 - val_acc: 0.6934
Epoch 6/10
3888/3888 [==============================] - 801s 206ms/step - loss: 0.0363 - acc: 0.9887 - val_loss: 2.2808 - val_acc: 0.7010
Epoch 7/10
3888/3888 [==============================] - 805s 207ms/step - loss: 0.0360 - acc: 0.9887 - val_loss: 2.8582 - val_acc: 0.6666
Epoch 8/10
3888/3888 [==============================] - 809s 208ms/step - loss: 0.0320 - acc: 0.9902 - val_loss: 2.7620 - val_acc: 0.6896
Epoch 9/10
3888/3888 [==============================] - 796s 205ms/step - loss: 0.0280 - acc: 0.9912 - val_loss: 2.7519 - val_acc: 0.6948
Epoch 10/10
3888/3888 [==============================] - 793s 204ms/step - loss: 0.0264 - acc: 0.9920 - val_loss: 3.0678 - val_acc: 0.6643
Out[61]:
<keras.callbacks.History at 0x7fc6759090f0>

The accuracy from this simple model is around 68% which is quite impressive. But still, the accuracy can be increased with a more complex model. Additionaly, this model has some serious overfitting problem.

CNN with VGG16 Model

[6]VGG16 was publised in 2014 and is one of the simplest. It is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper “Very Deep Convolutional Networks for Large-Scale Image Recognition”. The model achieves 92.7% top-5 test accuracy in ImageNet, which is a dataset of over 14 million images belonging to 1000 classes. It's Key Characteristics are:

  • This network contains total 16 layers in which weights and bias parameters are learnt.
  • A total of 13 convolutional layers are stacked one after the other and 3 dense layers for classification.
  • The number of filters in the convolution layers follow an increasing pattern (similar to decoder architecture of autoencoder).
  • The informative features are obtained by max pooling layers applied at different steps in the architecture.
  • The dense layers comprises of 4096, 4096, and 1000 nodes each.
  • The cons of this architecture are that it is slow to train and produces the model with very large size.

The VGG16 architecture is given below:

A pretrained VGG16 model can be downloaded from Keras. Unfortunately thre pretrained weights can not be used, since the input shape in this case is different. According to the Documentation, the weights will be randomly initialized. In addition to two dropout layers will be included in the model, to reduce overfitting and increase accuracy.

In [10]:
vgg16_model = keras.applications.vgg16.VGG16(input_shape=(80, 80, 3),classes=5, weights=None)


# Store the fully connected layers
fc1 = vgg16_model.layers[-3]
fc2 = vgg16_model.layers[-2]
predictions = vgg16_model.layers[-1]

# Create the dropout layers
dropout1 = Dropout(0.4)
dropout2 = Dropout(0.4)

# Reconnect the layers
model = Sequential()
for layer in vgg16_model.layers[:-3]:
    model.add(layer)

model.add(fc1)
model.add(dropout1)
model.add(fc2)
model.add(dropout2)
model.add(predictions)
In [11]:
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
block1_conv1 (Conv2D)        (None, 80, 80, 64)        1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 80, 80, 64)        36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 40, 40, 64)        0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 40, 40, 128)       73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 40, 40, 128)       147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 20, 20, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 20, 20, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 20, 20, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 20, 20, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 10, 10, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 10, 10, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 10, 10, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 10, 10, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 5, 5, 512)         0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 5, 5, 512)         2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 5, 5, 512)         2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 5, 5, 512)         2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 2, 2, 512)         0         
_________________________________________________________________
flatten (Flatten)            (None, 2048)              0         
_________________________________________________________________
fc1 (Dense)                  (None, 4096)              8392704   
_________________________________________________________________
dropout_3 (Dropout)          (None, 4096)              0         
_________________________________________________________________
fc2 (Dense)                  (None, 4096)              16781312  
_________________________________________________________________
dropout_4 (Dropout)          (None, 4096)              0         
_________________________________________________________________
predictions (Dense)          (None, 5)                 20485     
=================================================================
Total params: 39,909,189
Trainable params: 39,909,189
Non-trainable params: 0
_________________________________________________________________

The following steps have been taken to increase the accuracy from the basis VGG16 model with five training epochs (~70% accuracy):

  • target_size has been increased to 80x80 (increased accuracy of ~+2%)
  • batch_size has been decreased from 32 to 16 (increased accuracy of ~+2%)
  • epochs has been increased from 10 to 50 (increased accuracy of ~+2%, 50 epochs are a bit of an overkill)
  • learning rate has been set to 0.0001 (increased accuracy of ~+2%)
  • Dropout Layer has been added (increased accuracy of ~+2%) and decreased overfitting a lot

Each of these steps came with the downside of longer calculation time.

In [12]:
model.compile(keras.optimizers.Adam(lr=0.0001), loss = 'categorical_crossentropy', metrics = ['accuracy'])
In [14]:
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('./data/train',
                                                 target_size = (80, 80),
                                                 batch_size = 16)

test_set = test_datagen.flow_from_directory('./data/test',
                                            target_size = (80, 80),
                                            batch_size = 16)

history = model.fit_generator(training_set,
                         steps_per_epoch = 3888,
                         epochs = 50,
                         validation_data = test_set,
                         validation_steps = 435)
Found 3888 images belonging to 5 classes.
Found 435 images belonging to 5 classes.
Epoch 1/50
3888/3888 [==============================] - 607s 156ms/step - loss: 1.1017 - acc: 0.5412 - val_loss: 0.7718 - val_acc: 0.7314
Epoch 2/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.5850 - acc: 0.7866 - val_loss: 0.7445 - val_acc: 0.7721
Epoch 3/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.3128 - acc: 0.8869 - val_loss: 1.1667 - val_acc: 0.7361
Epoch 4/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.1654 - acc: 0.9431 - val_loss: 1.0530 - val_acc: 0.7539
Epoch 5/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.1021 - acc: 0.9661 - val_loss: 1.2316 - val_acc: 0.7555
Epoch 6/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0783 - acc: 0.9748 - val_loss: 1.2160 - val_acc: 0.7630
Epoch 7/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0638 - acc: 0.9795 - val_loss: 1.5365 - val_acc: 0.7660
Epoch 8/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0561 - acc: 0.9820 - val_loss: 1.0634 - val_acc: 0.8091
Epoch 9/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0467 - acc: 0.9850 - val_loss: 1.3274 - val_acc: 0.8049
Epoch 10/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0432 - acc: 0.9864 - val_loss: 1.8200 - val_acc: 0.7930
Epoch 11/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0438 - acc: 0.9867 - val_loss: 1.2985 - val_acc: 0.7978
Epoch 12/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0426 - acc: 0.9872 - val_loss: 1.3066 - val_acc: 0.7924
Epoch 13/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0321 - acc: 0.9901 - val_loss: 1.6710 - val_acc: 0.7898
Epoch 14/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0382 - acc: 0.9881 - val_loss: 1.3351 - val_acc: 0.7836
Epoch 15/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0359 - acc: 0.9890 - val_loss: 1.5741 - val_acc: 0.7870
Epoch 16/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0363 - acc: 0.9889 - val_loss: 1.2980 - val_acc: 0.7811
Epoch 17/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0320 - acc: 0.9904 - val_loss: 1.3659 - val_acc: 0.7940
Epoch 18/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0328 - acc: 0.9905 - val_loss: 1.3876 - val_acc: 0.7947
Epoch 19/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0302 - acc: 0.9912 - val_loss: 1.8996 - val_acc: 0.8017
Epoch 20/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0314 - acc: 0.9910 - val_loss: 1.4051 - val_acc: 0.7941
Epoch 21/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0324 - acc: 0.9908 - val_loss: 1.6868 - val_acc: 0.7913
Epoch 22/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0275 - acc: 0.9917 - val_loss: 1.3973 - val_acc: 0.7911
Epoch 23/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0308 - acc: 0.9912 - val_loss: 1.2626 - val_acc: 0.7860
Epoch 24/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0287 - acc: 0.9915 - val_loss: 1.3890 - val_acc: 0.7904
Epoch 25/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0288 - acc: 0.9914 - val_loss: 1.6075 - val_acc: 0.8073
Epoch 26/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0308 - acc: 0.9911 - val_loss: 1.6209 - val_acc: 0.8132
Epoch 27/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0313 - acc: 0.9920 - val_loss: 1.2223 - val_acc: 0.7956
Epoch 28/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0280 - acc: 0.9922 - val_loss: 1.4885 - val_acc: 0.7863
Epoch 29/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0318 - acc: 0.9914 - val_loss: 1.3095 - val_acc: 0.8120
Epoch 30/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0312 - acc: 0.9919 - val_loss: 1.6056 - val_acc: 0.8113
Epoch 31/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0312 - acc: 0.9919 - val_loss: 1.4461 - val_acc: 0.8003
Epoch 32/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0280 - acc: 0.9928 - val_loss: 1.5034 - val_acc: 0.7827
Epoch 33/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0298 - acc: 0.9925 - val_loss: 1.5074 - val_acc: 0.8049
Epoch 34/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0385 - acc: 0.9912 - val_loss: 1.8617 - val_acc: 0.7975
Epoch 35/50
3888/3888 [==============================] - 605s 156ms/step - loss: 0.0267 - acc: 0.9931 - val_loss: 1.9946 - val_acc: 0.8009
Epoch 36/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0292 - acc: 0.9922 - val_loss: 1.1370 - val_acc: 0.7756
Epoch 37/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0295 - acc: 0.9925 - val_loss: 1.7840 - val_acc: 0.7920
Epoch 38/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0285 - acc: 0.9928 - val_loss: 1.6622 - val_acc: 0.7977
Epoch 39/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0366 - acc: 0.9910 - val_loss: 1.3964 - val_acc: 0.8022
Epoch 40/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0246 - acc: 0.9934 - val_loss: 1.4968 - val_acc: 0.8109
Epoch 41/50
3888/3888 [==============================] - 605s 156ms/step - loss: 0.0259 - acc: 0.9935 - val_loss: 1.5783 - val_acc: 0.7425
Epoch 42/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0466 - acc: 0.9903 - val_loss: 1.8798 - val_acc: 0.7953
Epoch 43/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0673 - acc: 0.9800 - val_loss: 1.7591 - val_acc: 0.7922
Epoch 44/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0274 - acc: 0.9937 - val_loss: 1.3350 - val_acc: 0.8235
Epoch 45/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0311 - acc: 0.9926 - val_loss: 1.4718 - val_acc: 0.7810
Epoch 46/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0329 - acc: 0.9918 - val_loss: 1.7229 - val_acc: 0.7846
Epoch 47/50
3888/3888 [==============================] - 606s 156ms/step - loss: 0.0346 - acc: 0.9919 - val_loss: 1.8502 - val_acc: 0.8049
Epoch 48/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0369 - acc: 0.9914 - val_loss: 1.6997 - val_acc: 0.7975
Epoch 49/50
3888/3888 [==============================] - 607s 156ms/step - loss: 0.0334 - acc: 0.9927 - val_loss: 1.5077 - val_acc: 0.8303
Epoch 50/50
3888/3888 [==============================] - 608s 156ms/step - loss: 0.0382 - acc: 0.9910 - val_loss: 1.3917 - val_acc: 0.8126
In [23]:
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()
In [25]:
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()

Improving the model

Furthermore the model still can be improved:

  • The model receives batch wise
  • ReduceLROnPlateau will be used. It reduces learning rate when a metric has stopped improving. Models often benefit from reducing the learning rate by a factor of 2-10 once learning stagnates. This callback monitors a quantity and if no improvement is seen for a 'patience' number of epochs, the learning rate is reduced.
  • Bigger input_shape of the pictures, with the knowledge, that some smaller pictures will lose quality.
In [11]:
vgg16_model = keras.applications.vgg16.VGG16(input_shape=(150, 150, 3),classes=5, weights=None)

# Store the fully connected layers
fc1 = vgg16_model.layers[-3]
fc2 = vgg16_model.layers[-2]
predictions = vgg16_model.layers[-1]

# Create the dropout layers
dropout1 = Dropout(0.4)
dropout2 = Dropout(0.4)

# Reconnect the layers
model2 = Sequential()
for layer in vgg16_model.layers[:-3]:
    model2.add(layer)

model2.add(fc1)
model2.add(dropout1)
model2.add(fc2)
model2.add(dropout2)
model2.add(predictions)
In [12]:
model2.compile(keras.optimizers.Adam(lr=0.0001), loss = 'categorical_crossentropy', metrics = ['accuracy'])
In [13]:
from keras.callbacks import ReduceLROnPlateau

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.5, 
                                            min_lr=0.000001)


train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('./data/train',
                                                 target_size = (150, 150),
                                                 batch_size = 16)

test_set = test_datagen.flow_from_directory('./data/test',
                                            target_size = (150, 150),
                                            batch_size = 16)

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', patience=3, verbose=1, factor=0.5, min_lr=2.5e-5)

history = model2.fit_generator(training_set,
                         steps_per_epoch = 3888 //16,
                         epochs = 100,
                         validation_data = test_set,
                         validation_steps = 435 // 16,
                         callbacks=[learning_rate_reduction])
Found 3888 images belonging to 5 classes.
Found 435 images belonging to 5 classes.
Epoch 1/100
243/243 [==============================] - 90s 370ms/step - loss: 1.4958 - acc: 0.3107 - val_loss: 1.2962 - val_acc: 0.4375
Epoch 2/100
243/243 [==============================] - 80s 331ms/step - loss: 1.3084 - acc: 0.4108 - val_loss: 1.3019 - val_acc: 0.4344
Epoch 3/100
243/243 [==============================] - 80s 328ms/step - loss: 1.2745 - acc: 0.4257 - val_loss: 1.2185 - val_acc: 0.4344
Epoch 4/100
243/243 [==============================] - 80s 328ms/step - loss: 1.2532 - acc: 0.4493 - val_loss: 1.1435 - val_acc: 0.4749
Epoch 5/100
243/243 [==============================] - 80s 328ms/step - loss: 1.1833 - acc: 0.4985 - val_loss: 1.2128 - val_acc: 0.4988
Epoch 6/100
243/243 [==============================] - 80s 328ms/step - loss: 1.1076 - acc: 0.5517 - val_loss: 1.0635 - val_acc: 0.5728
Epoch 7/100
243/243 [==============================] - 80s 328ms/step - loss: 1.0495 - acc: 0.5910 - val_loss: 1.0015 - val_acc: 0.5800
Epoch 8/100
243/243 [==============================] - 80s 328ms/step - loss: 0.9499 - acc: 0.6386 - val_loss: 0.9394 - val_acc: 0.6110
Epoch 9/100
243/243 [==============================] - 80s 328ms/step - loss: 0.9209 - acc: 0.6487 - val_loss: 0.8413 - val_acc: 0.6993
Epoch 10/100
243/243 [==============================] - 80s 328ms/step - loss: 0.8832 - acc: 0.6662 - val_loss: 0.8372 - val_acc: 0.7088
Epoch 11/100
243/243 [==============================] - 80s 328ms/step - loss: 0.8199 - acc: 0.6970 - val_loss: 0.8599 - val_acc: 0.6778
Epoch 12/100
243/243 [==============================] - 80s 328ms/step - loss: 0.7806 - acc: 0.7137 - val_loss: 0.6814 - val_acc: 0.7518
Epoch 13/100
243/243 [==============================] - 80s 328ms/step - loss: 0.7455 - acc: 0.7189 - val_loss: 0.8767 - val_acc: 0.6850
Epoch 14/100
243/243 [==============================] - 80s 328ms/step - loss: 0.7256 - acc: 0.7346 - val_loss: 0.7208 - val_acc: 0.7375
Epoch 15/100
243/243 [==============================] - 80s 328ms/step - loss: 0.7012 - acc: 0.7428 - val_loss: 0.7719 - val_acc: 0.7112

Epoch 00015: ReduceLROnPlateau reducing learning rate to 4.999999873689376e-05.
Epoch 16/100
243/243 [==============================] - 80s 328ms/step - loss: 0.6251 - acc: 0.7657 - val_loss: 0.7815 - val_acc: 0.7255
Epoch 17/100
243/243 [==============================] - 80s 328ms/step - loss: 0.5952 - acc: 0.7778 - val_loss: 0.7259 - val_acc: 0.7613
Epoch 18/100
243/243 [==============================] - 80s 328ms/step - loss: 0.5666 - acc: 0.7909 - val_loss: 0.6155 - val_acc: 0.7947
Epoch 19/100
243/243 [==============================] - 80s 328ms/step - loss: 0.5626 - acc: 0.7973 - val_loss: 0.7659 - val_acc: 0.7446
Epoch 20/100
243/243 [==============================] - 80s 328ms/step - loss: 0.5257 - acc: 0.8066 - val_loss: 0.6080 - val_acc: 0.7685
Epoch 21/100
243/243 [==============================] - 80s 328ms/step - loss: 0.5240 - acc: 0.8004 - val_loss: 0.6959 - val_acc: 0.7470

Epoch 00021: ReduceLROnPlateau reducing learning rate to 2.5e-05.
Epoch 22/100
243/243 [==============================] - 80s 328ms/step - loss: 0.4452 - acc: 0.8377 - val_loss: 0.7184 - val_acc: 0.7876
Epoch 23/100
243/243 [==============================] - 80s 328ms/step - loss: 0.4276 - acc: 0.8449 - val_loss: 0.7184 - val_acc: 0.7828
Epoch 24/100
243/243 [==============================] - 80s 328ms/step - loss: 0.4087 - acc: 0.8503 - val_loss: 0.7748 - val_acc: 0.7375
Epoch 25/100
243/243 [==============================] - 80s 328ms/step - loss: 0.3888 - acc: 0.8585 - val_loss: 0.7330 - val_acc: 0.7924
Epoch 26/100
243/243 [==============================] - 80s 328ms/step - loss: 0.3822 - acc: 0.8588 - val_loss: 0.7503 - val_acc: 0.7780
Epoch 27/100
243/243 [==============================] - 80s 328ms/step - loss: 0.3522 - acc: 0.8665 - val_loss: 0.7499 - val_acc: 0.7804
Epoch 28/100
243/243 [==============================] - 80s 328ms/step - loss: 0.3388 - acc: 0.8801 - val_loss: 0.8278 - val_acc: 0.7780
Epoch 29/100
243/243 [==============================] - 80s 329ms/step - loss: 0.3274 - acc: 0.8837 - val_loss: 0.7816 - val_acc: 0.7523
Epoch 30/100
243/243 [==============================] - 80s 328ms/step - loss: 0.3090 - acc: 0.8927 - val_loss: 0.7888 - val_acc: 0.7876
Epoch 31/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2929 - acc: 0.8966 - val_loss: 0.8408 - val_acc: 0.7446
Epoch 32/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2764 - acc: 0.8979 - val_loss: 0.8792 - val_acc: 0.7709
Epoch 33/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2582 - acc: 0.9097 - val_loss: 0.7348 - val_acc: 0.7971
Epoch 34/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2618 - acc: 0.9061 - val_loss: 1.0109 - val_acc: 0.7900
Epoch 35/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2315 - acc: 0.9177 - val_loss: 0.8660 - val_acc: 0.7685
Epoch 36/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2337 - acc: 0.9195 - val_loss: 0.9573 - val_acc: 0.7685
Epoch 37/100
243/243 [==============================] - 80s 328ms/step - loss: 0.2227 - acc: 0.9162 - val_loss: 1.0318 - val_acc: 0.7589
Epoch 38/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1968 - acc: 0.9303 - val_loss: 0.9957 - val_acc: 0.7566
Epoch 39/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1852 - acc: 0.9347 - val_loss: 1.0243 - val_acc: 0.7828
Epoch 40/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1694 - acc: 0.9388 - val_loss: 0.9933 - val_acc: 0.7470
Epoch 41/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1888 - acc: 0.9324 - val_loss: 1.2797 - val_acc: 0.7566
Epoch 42/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1604 - acc: 0.9398 - val_loss: 0.9748 - val_acc: 0.7637
Epoch 43/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1589 - acc: 0.9460 - val_loss: 1.1194 - val_acc: 0.7566
Epoch 44/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1525 - acc: 0.9465 - val_loss: 1.1724 - val_acc: 0.7589
Epoch 45/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1327 - acc: 0.9547 - val_loss: 1.0809 - val_acc: 0.7924
Epoch 46/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1430 - acc: 0.9547 - val_loss: 1.1527 - val_acc: 0.7542
Epoch 47/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1242 - acc: 0.9570 - val_loss: 1.1285 - val_acc: 0.7852
Epoch 48/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1243 - acc: 0.9555 - val_loss: 1.0955 - val_acc: 0.7804
Epoch 49/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1078 - acc: 0.9630 - val_loss: 1.1982 - val_acc: 0.7733
Epoch 50/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1197 - acc: 0.9586 - val_loss: 1.3915 - val_acc: 0.7566
Epoch 51/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1273 - acc: 0.9570 - val_loss: 1.1068 - val_acc: 0.8019
Epoch 52/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1111 - acc: 0.9632 - val_loss: 1.1700 - val_acc: 0.7518
Epoch 53/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0933 - acc: 0.9681 - val_loss: 1.1292 - val_acc: 0.7852
Epoch 54/100
243/243 [==============================] - 80s 328ms/step - loss: 0.1050 - acc: 0.9663 - val_loss: 1.2025 - val_acc: 0.7924
Epoch 55/100
243/243 [==============================] - 80s 329ms/step - loss: 0.0848 - acc: 0.9676 - val_loss: 1.2514 - val_acc: 0.7518
Epoch 56/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0861 - acc: 0.9722 - val_loss: 1.4171 - val_acc: 0.7566
Epoch 57/100
243/243 [==============================] - 80s 329ms/step - loss: 0.1085 - acc: 0.9648 - val_loss: 1.2601 - val_acc: 0.7731
Epoch 58/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0823 - acc: 0.9738 - val_loss: 1.2081 - val_acc: 0.7589
Epoch 59/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0799 - acc: 0.9730 - val_loss: 1.3334 - val_acc: 0.7661
Epoch 60/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0759 - acc: 0.9774 - val_loss: 1.1342 - val_acc: 0.7876
Epoch 61/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0842 - acc: 0.9717 - val_loss: 1.3729 - val_acc: 0.7637
Epoch 62/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0785 - acc: 0.9756 - val_loss: 1.2757 - val_acc: 0.7804
Epoch 63/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0942 - acc: 0.9689 - val_loss: 1.2489 - val_acc: 0.7947
Epoch 64/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0822 - acc: 0.9751 - val_loss: 1.2567 - val_acc: 0.7518
Epoch 65/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0613 - acc: 0.9817 - val_loss: 1.1624 - val_acc: 0.8067
Epoch 66/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0817 - acc: 0.9735 - val_loss: 1.1543 - val_acc: 0.7947
Epoch 67/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0646 - acc: 0.9797 - val_loss: 1.6062 - val_acc: 0.7255
Epoch 68/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0548 - acc: 0.9815 - val_loss: 1.1684 - val_acc: 0.7828
Epoch 69/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0471 - acc: 0.9856 - val_loss: 1.2694 - val_acc: 0.7995
Epoch 70/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0699 - acc: 0.9776 - val_loss: 1.2757 - val_acc: 0.7924
Epoch 71/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0706 - acc: 0.9753 - val_loss: 1.3815 - val_acc: 0.7542
Epoch 72/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0589 - acc: 0.9817 - val_loss: 1.5121 - val_acc: 0.7446
Epoch 73/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0589 - acc: 0.9784 - val_loss: 1.4715 - val_acc: 0.7375
Epoch 74/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0526 - acc: 0.9838 - val_loss: 1.0906 - val_acc: 0.8186
Epoch 75/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0705 - acc: 0.9771 - val_loss: 1.3396 - val_acc: 0.7852
Epoch 76/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0453 - acc: 0.9869 - val_loss: 1.3744 - val_acc: 0.7470
Epoch 77/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0444 - acc: 0.9866 - val_loss: 1.3333 - val_acc: 0.7828
Epoch 78/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0451 - acc: 0.9864 - val_loss: 1.2429 - val_acc: 0.8115
Epoch 79/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0661 - acc: 0.9769 - val_loss: 1.7080 - val_acc: 0.7566
Epoch 80/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0493 - acc: 0.9815 - val_loss: 1.2025 - val_acc: 0.7971
Epoch 81/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0475 - acc: 0.9859 - val_loss: 1.4522 - val_acc: 0.7685
Epoch 82/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0559 - acc: 0.9833 - val_loss: 1.4034 - val_acc: 0.7757
Epoch 83/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0294 - acc: 0.9920 - val_loss: 1.4733 - val_acc: 0.7780
Epoch 84/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0642 - acc: 0.9779 - val_loss: 1.3376 - val_acc: 0.7613
Epoch 85/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0336 - acc: 0.9887 - val_loss: 1.4083 - val_acc: 0.7639
Epoch 86/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0335 - acc: 0.9900 - val_loss: 1.3311 - val_acc: 0.7828
Epoch 87/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0346 - acc: 0.9884 - val_loss: 1.3552 - val_acc: 0.8043
Epoch 88/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0440 - acc: 0.9856 - val_loss: 1.5129 - val_acc: 0.7924
Epoch 89/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0550 - acc: 0.9828 - val_loss: 1.4135 - val_acc: 0.7518
Epoch 90/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0396 - acc: 0.9884 - val_loss: 1.5896 - val_acc: 0.7685
Epoch 91/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0451 - acc: 0.9838 - val_loss: 1.3973 - val_acc: 0.7709
Epoch 92/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0401 - acc: 0.9877 - val_loss: 1.5850 - val_acc: 0.7733
Epoch 93/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0489 - acc: 0.9879 - val_loss: 1.4019 - val_acc: 0.7637
Epoch 94/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0454 - acc: 0.9869 - val_loss: 1.4285 - val_acc: 0.7876
Epoch 95/100
243/243 [==============================] - 80s 329ms/step - loss: 0.0376 - acc: 0.9874 - val_loss: 1.4922 - val_acc: 0.7589
Epoch 96/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0209 - acc: 0.9925 - val_loss: 1.4893 - val_acc: 0.7828
Epoch 97/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0271 - acc: 0.9902 - val_loss: 1.5027 - val_acc: 0.7470
Epoch 98/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0466 - acc: 0.9859 - val_loss: 1.4017 - val_acc: 0.8115
Epoch 99/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0375 - acc: 0.9892 - val_loss: 1.4460 - val_acc: 0.7685
Epoch 100/100
243/243 [==============================] - 80s 328ms/step - loss: 0.0386 - acc: 0.9864 - val_loss: 1.3832 - val_acc: 0.7733
In [14]:
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()
In [15]:
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()

As it can bee seen, the newer version of the VGG16 model didn't improved and has the same model accuracy. Therefore a new model will be tried. The new model is the ResNet50.

In [5]:
from keras.applications.resnet50 import ResNet50
In [8]:
resnet50_model = ResNet50(include_top=True, weights=None, input_shape=(150, 150, 3), classes = 5)
In [9]:
resnet50_model.compile(keras.optimizers.Adam(lr=0.0001), loss = 'categorical_crossentropy', metrics = ['accuracy'])

from keras.callbacks import ReduceLROnPlateau

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', 
                                            patience=3, 
                                            verbose=1, 
                                            factor=0.5, 
                                            min_lr=0.000001)


train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('./data/train',
                                                 target_size = (150, 150),
                                                 batch_size = 16)

test_set = test_datagen.flow_from_directory('./data/test',
                                            target_size = (150, 150),
                                            batch_size = 16)

learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', patience=3, verbose=1, factor=0.5, min_lr=2.5e-5)

history = resnet50_model.fit_generator(training_set,
                         steps_per_epoch = 3888 //16,
                         epochs = 100,
                         validation_data = test_set,
                         validation_steps = 435 // 16,
                         callbacks=[learning_rate_reduction])
Found 3888 images belonging to 5 classes.
Found 435 images belonging to 5 classes.
Epoch 1/100
243/243 [==============================] - 119s 491ms/step - loss: 1.4375 - acc: 0.4164 - val_loss: 3.8128 - val_acc: 0.2269
Epoch 2/100
243/243 [==============================] - 96s 394ms/step - loss: 1.2383 - acc: 0.4949 - val_loss: 2.0891 - val_acc: 0.4224
Epoch 3/100
243/243 [==============================] - 96s 393ms/step - loss: 1.1617 - acc: 0.5283 - val_loss: 1.3277 - val_acc: 0.4916
Epoch 4/100
243/243 [==============================] - 95s 393ms/step - loss: 1.1148 - acc: 0.5540 - val_loss: 1.1006 - val_acc: 0.5704
Epoch 5/100
243/243 [==============================] - 96s 396ms/step - loss: 1.1005 - acc: 0.5556 - val_loss: 1.3123 - val_acc: 0.5632
Epoch 6/100
243/243 [==============================] - 96s 394ms/step - loss: 1.0464 - acc: 0.5864 - val_loss: 1.4699 - val_acc: 0.5776
Epoch 7/100
243/243 [==============================] - 95s 391ms/step - loss: 1.0408 - acc: 0.5880 - val_loss: 1.7033 - val_acc: 0.5179
Epoch 8/100
243/243 [==============================] - 95s 392ms/step - loss: 0.9669 - acc: 0.6227 - val_loss: 1.1818 - val_acc: 0.5561
Epoch 9/100
243/243 [==============================] - 95s 393ms/step - loss: 0.9465 - acc: 0.6217 - val_loss: 1.0944 - val_acc: 0.6038
Epoch 10/100
243/243 [==============================] - 96s 393ms/step - loss: 0.9382 - acc: 0.6322 - val_loss: 1.2434 - val_acc: 0.6277
Epoch 11/100
243/243 [==============================] - 96s 393ms/step - loss: 0.9097 - acc: 0.6458 - val_loss: 0.9834 - val_acc: 0.6492
Epoch 12/100
243/243 [==============================] - 96s 394ms/step - loss: 0.8860 - acc: 0.6579 - val_loss: 1.4639 - val_acc: 0.5442
Epoch 13/100
243/243 [==============================] - 95s 393ms/step - loss: 0.8514 - acc: 0.6646 - val_loss: 1.1577 - val_acc: 0.5871
Epoch 14/100
243/243 [==============================] - 95s 391ms/step - loss: 0.8244 - acc: 0.6844 - val_loss: 0.8358 - val_acc: 0.6683
Epoch 15/100
243/243 [==============================] - 95s 391ms/step - loss: 0.7923 - acc: 0.6955 - val_loss: 1.1797 - val_acc: 0.6134
Epoch 16/100
243/243 [==============================] - 95s 392ms/step - loss: 0.7756 - acc: 0.7016 - val_loss: 0.9444 - val_acc: 0.6659
Epoch 17/100
243/243 [==============================] - 95s 392ms/step - loss: 0.7516 - acc: 0.7181 - val_loss: 1.2326 - val_acc: 0.6134

Epoch 00017: ReduceLROnPlateau reducing learning rate to 4.999999873689376e-05.
Epoch 18/100
243/243 [==============================] - 95s 393ms/step - loss: 0.6783 - acc: 0.7361 - val_loss: 0.7927 - val_acc: 0.7184
Epoch 19/100
243/243 [==============================] - 96s 393ms/step - loss: 0.6207 - acc: 0.7683 - val_loss: 0.7867 - val_acc: 0.7351
Epoch 20/100
243/243 [==============================] - 96s 395ms/step - loss: 0.6110 - acc: 0.7698 - val_loss: 0.7802 - val_acc: 0.7064
Epoch 21/100
243/243 [==============================] - 96s 396ms/step - loss: 0.5954 - acc: 0.7762 - val_loss: 0.8095 - val_acc: 0.7303
Epoch 22/100
243/243 [==============================] - 96s 393ms/step - loss: 0.5743 - acc: 0.7814 - val_loss: 0.8453 - val_acc: 0.7041

Epoch 00022: ReduceLROnPlateau reducing learning rate to 2.5e-05.
Epoch 23/100
243/243 [==============================] - 95s 393ms/step - loss: 0.5058 - acc: 0.8107 - val_loss: 0.8074 - val_acc: 0.7017
Epoch 24/100
243/243 [==============================] - 96s 397ms/step - loss: 0.4721 - acc: 0.8272 - val_loss: 0.7164 - val_acc: 0.7685
Epoch 25/100
243/243 [==============================] - 96s 395ms/step - loss: 0.4890 - acc: 0.8223 - val_loss: 0.7453 - val_acc: 0.7446
Epoch 26/100
243/243 [==============================] - 96s 397ms/step - loss: 0.4848 - acc: 0.8238 - val_loss: 0.7516 - val_acc: 0.7399
Epoch 27/100
243/243 [==============================] - 96s 395ms/step - loss: 0.4509 - acc: 0.8351 - val_loss: 0.7791 - val_acc: 0.7327
Epoch 28/100
243/243 [==============================] - 96s 395ms/step - loss: 0.4336 - acc: 0.8423 - val_loss: 0.7784 - val_acc: 0.7542
Epoch 29/100
243/243 [==============================] - 96s 394ms/step - loss: 0.4199 - acc: 0.8485 - val_loss: 0.6712 - val_acc: 0.7708
Epoch 30/100
243/243 [==============================] - 96s 397ms/step - loss: 0.4143 - acc: 0.8516 - val_loss: 0.9050 - val_acc: 0.7208
Epoch 31/100
243/243 [==============================] - 95s 393ms/step - loss: 0.4053 - acc: 0.8526 - val_loss: 0.7828 - val_acc: 0.7327
Epoch 32/100
243/243 [==============================] - 96s 394ms/step - loss: 0.3822 - acc: 0.8603 - val_loss: 0.9017 - val_acc: 0.7208
Epoch 33/100
243/243 [==============================] - 96s 396ms/step - loss: 0.3814 - acc: 0.8606 - val_loss: 0.8829 - val_acc: 0.7279
Epoch 34/100
243/243 [==============================] - 96s 395ms/step - loss: 0.3678 - acc: 0.8652 - val_loss: 0.7017 - val_acc: 0.7422
Epoch 35/100
243/243 [==============================] - 96s 395ms/step - loss: 0.3508 - acc: 0.8740 - val_loss: 0.8973 - val_acc: 0.7375
Epoch 36/100
243/243 [==============================] - 97s 399ms/step - loss: 0.3177 - acc: 0.8863 - val_loss: 0.8233 - val_acc: 0.7399
Epoch 37/100
243/243 [==============================] - 97s 397ms/step - loss: 0.3078 - acc: 0.8884 - val_loss: 0.7360 - val_acc: 0.7589
Epoch 38/100
243/243 [==============================] - 96s 396ms/step - loss: 0.3105 - acc: 0.8879 - val_loss: 0.7635 - val_acc: 0.7303
Epoch 39/100
243/243 [==============================] - 97s 401ms/step - loss: 0.3169 - acc: 0.8879 - val_loss: 0.8181 - val_acc: 0.7303
Epoch 40/100
243/243 [==============================] - 98s 403ms/step - loss: 0.3002 - acc: 0.8897 - val_loss: 0.8287 - val_acc: 0.7422
Epoch 41/100
243/243 [==============================] - 96s 395ms/step - loss: 0.2943 - acc: 0.8930 - val_loss: 0.7590 - val_acc: 0.7685
Epoch 42/100
243/243 [==============================] - 96s 394ms/step - loss: 0.2782 - acc: 0.8994 - val_loss: 0.9676 - val_acc: 0.7160
Epoch 43/100
243/243 [==============================] - 97s 399ms/step - loss: 0.2708 - acc: 0.9041 - val_loss: 1.0870 - val_acc: 0.6874
Epoch 44/100
243/243 [==============================] - 97s 399ms/step - loss: 0.2532 - acc: 0.9128 - val_loss: 0.8713 - val_acc: 0.7327
Epoch 45/100
243/243 [==============================] - 97s 400ms/step - loss: 0.2509 - acc: 0.9074 - val_loss: 0.8133 - val_acc: 0.7160
Epoch 46/100
243/243 [==============================] - 97s 400ms/step - loss: 0.2321 - acc: 0.9164 - val_loss: 0.9304 - val_acc: 0.7542
Epoch 47/100
243/243 [==============================] - 96s 395ms/step - loss: 0.2413 - acc: 0.9126 - val_loss: 0.9248 - val_acc: 0.7064
Epoch 48/100
243/243 [==============================] - 95s 392ms/step - loss: 0.2221 - acc: 0.9210 - val_loss: 0.9529 - val_acc: 0.7232
Epoch 49/100
243/243 [==============================] - 97s 398ms/step - loss: 0.2204 - acc: 0.9203 - val_loss: 0.9150 - val_acc: 0.7375
Epoch 50/100
243/243 [==============================] - 97s 399ms/step - loss: 0.2202 - acc: 0.9210 - val_loss: 0.8861 - val_acc: 0.7470
Epoch 51/100
243/243 [==============================] - 96s 396ms/step - loss: 0.1955 - acc: 0.9318 - val_loss: 1.0619 - val_acc: 0.7351
Epoch 52/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1917 - acc: 0.9344 - val_loss: 0.9887 - val_acc: 0.7303
Epoch 53/100
243/243 [==============================] - 97s 400ms/step - loss: 0.2062 - acc: 0.9311 - val_loss: 0.8971 - val_acc: 0.7399
Epoch 54/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1710 - acc: 0.9365 - val_loss: 0.9013 - val_acc: 0.7566
Epoch 55/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1776 - acc: 0.9378 - val_loss: 0.8141 - val_acc: 0.7733
Epoch 56/100
243/243 [==============================] - 96s 397ms/step - loss: 0.1687 - acc: 0.9460 - val_loss: 1.0393 - val_acc: 0.7255
Epoch 57/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1758 - acc: 0.9388 - val_loss: 1.0517 - val_acc: 0.7153
Epoch 58/100
243/243 [==============================] - 96s 395ms/step - loss: 0.1697 - acc: 0.9424 - val_loss: 0.9832 - val_acc: 0.7399
Epoch 59/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1537 - acc: 0.9465 - val_loss: 0.9481 - val_acc: 0.7279
Epoch 60/100
243/243 [==============================] - 97s 397ms/step - loss: 0.1645 - acc: 0.9414 - val_loss: 1.0691 - val_acc: 0.7017
Epoch 61/100
243/243 [==============================] - 98s 403ms/step - loss: 0.1566 - acc: 0.9455 - val_loss: 0.8590 - val_acc: 0.7780
Epoch 62/100
243/243 [==============================] - 97s 398ms/step - loss: 0.1327 - acc: 0.9519 - val_loss: 0.9553 - val_acc: 0.7661
Epoch 63/100
243/243 [==============================] - 97s 398ms/step - loss: 0.1522 - acc: 0.9457 - val_loss: 1.1495 - val_acc: 0.7088
Epoch 64/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1560 - acc: 0.9455 - val_loss: 0.9841 - val_acc: 0.7470
Epoch 65/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1287 - acc: 0.9524 - val_loss: 1.0033 - val_acc: 0.7470
Epoch 66/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1389 - acc: 0.9504 - val_loss: 0.9727 - val_acc: 0.7637
Epoch 67/100
243/243 [==============================] - 98s 402ms/step - loss: 0.1208 - acc: 0.9570 - val_loss: 1.0053 - val_acc: 0.7709
Epoch 68/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1297 - acc: 0.9583 - val_loss: 1.3639 - val_acc: 0.6802
Epoch 69/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1089 - acc: 0.9614 - val_loss: 1.3124 - val_acc: 0.7064
Epoch 70/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1209 - acc: 0.9581 - val_loss: 0.9721 - val_acc: 0.7422
Epoch 71/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1168 - acc: 0.9586 - val_loss: 1.0219 - val_acc: 0.7542
Epoch 72/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1266 - acc: 0.9558 - val_loss: 0.8635 - val_acc: 0.7733
Epoch 73/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1220 - acc: 0.9604 - val_loss: 1.4182 - val_acc: 0.7303
Epoch 74/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1117 - acc: 0.9630 - val_loss: 0.8854 - val_acc: 0.7709
Epoch 75/100
243/243 [==============================] - 97s 401ms/step - loss: 0.1202 - acc: 0.9565 - val_loss: 1.0991 - val_acc: 0.7136
Epoch 76/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1078 - acc: 0.9653 - val_loss: 1.0453 - val_acc: 0.7542
Epoch 77/100
243/243 [==============================] - 97s 401ms/step - loss: 0.1027 - acc: 0.9645 - val_loss: 1.2074 - val_acc: 0.7088
Epoch 78/100
243/243 [==============================] - 97s 401ms/step - loss: 0.1005 - acc: 0.9673 - val_loss: 1.0522 - val_acc: 0.7399
Epoch 79/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1009 - acc: 0.9658 - val_loss: 1.1120 - val_acc: 0.7589
Epoch 80/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1201 - acc: 0.9537 - val_loss: 1.0073 - val_acc: 0.7709
Epoch 81/100
243/243 [==============================] - 97s 401ms/step - loss: 0.0950 - acc: 0.9686 - val_loss: 0.9848 - val_acc: 0.7685
Epoch 82/100
243/243 [==============================] - 97s 399ms/step - loss: 0.1015 - acc: 0.9637 - val_loss: 1.0319 - val_acc: 0.7422
Epoch 83/100
243/243 [==============================] - 97s 401ms/step - loss: 0.0977 - acc: 0.9668 - val_loss: 1.1251 - val_acc: 0.7422
Epoch 84/100
243/243 [==============================] - 97s 401ms/step - loss: 0.1042 - acc: 0.9655 - val_loss: 0.9703 - val_acc: 0.7494
Epoch 85/100
243/243 [==============================] - 97s 400ms/step - loss: 0.1002 - acc: 0.9668 - val_loss: 1.0506 - val_acc: 0.7477
Epoch 86/100
243/243 [==============================] - 97s 401ms/step - loss: 0.0975 - acc: 0.9681 - val_loss: 1.1658 - val_acc: 0.7255
Epoch 87/100
243/243 [==============================] - 97s 401ms/step - loss: 0.0798 - acc: 0.9727 - val_loss: 1.3609 - val_acc: 0.7184
Epoch 88/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0808 - acc: 0.9727 - val_loss: 1.1493 - val_acc: 0.7494
Epoch 89/100
243/243 [==============================] - 97s 399ms/step - loss: 0.0842 - acc: 0.9697 - val_loss: 0.9841 - val_acc: 0.7375
Epoch 90/100
243/243 [==============================] - 98s 404ms/step - loss: 0.0885 - acc: 0.9689 - val_loss: 1.3268 - val_acc: 0.7399
Epoch 91/100
243/243 [==============================] - 97s 399ms/step - loss: 0.0827 - acc: 0.9709 - val_loss: 0.8986 - val_acc: 0.7828
Epoch 92/100
243/243 [==============================] - 98s 402ms/step - loss: 0.0810 - acc: 0.9709 - val_loss: 1.1875 - val_acc: 0.7279
Epoch 93/100
243/243 [==============================] - 98s 402ms/step - loss: 0.0989 - acc: 0.9689 - val_loss: 1.2724 - val_acc: 0.7136
Epoch 94/100
243/243 [==============================] - 98s 402ms/step - loss: 0.0825 - acc: 0.9720 - val_loss: 1.1073 - val_acc: 0.7422
Epoch 95/100
243/243 [==============================] - 97s 401ms/step - loss: 0.0776 - acc: 0.9735 - val_loss: 1.1997 - val_acc: 0.7542
Epoch 96/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0713 - acc: 0.9789 - val_loss: 1.0073 - val_acc: 0.7661
Epoch 97/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0810 - acc: 0.9745 - val_loss: 1.0115 - val_acc: 0.7685
Epoch 98/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0776 - acc: 0.9763 - val_loss: 1.2957 - val_acc: 0.7399
Epoch 99/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0836 - acc: 0.9733 - val_loss: 1.1273 - val_acc: 0.7542
Epoch 100/100
243/243 [==============================] - 97s 400ms/step - loss: 0.0683 - acc: 0.9776 - val_loss: 1.3497 - val_acc: 0.7232
In [10]:
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()
In [11]:
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epochs')
plt.legend(['train', 'test'])
plt.show()

Unfortunately, the accuracy on the validation set hasn't changed. Therefore, it will be investigated which plant species had the most problems.

In [222]:
test_set = test_datagen.flow_from_directory('./data/test',
                                            target_size = (150, 150),
                                            batch_size = 435,
                                            shuffle = False)
Found 435 images belonging to 5 classes.
In [133]:
## copied from source[7]

predictions = resnet50_model.predict_generator(generator=test_set, steps=1)
y_pred = [np.argmax(probas) for probas in predictions]
y_test = test_set.classes
class_names = test_set.class_indices.keys()

from sklearn.metrics import confusion_matrix
import itertools

def plot_confusion_matrix(cm, classes, title='Confusion matrix', cmap=plt.cm.Blues):
    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
    plt.figure(figsize=(10,10))
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    fmt = '.2f'
    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, format(cm[i, j], fmt),
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.ylabel('True label')
    plt.xlabel('Predicted label')
    plt.tight_layout()
    
# compute confusion matrix
cnf_matrix = confusion_matrix(y_test, y_pred)
np.set_printoptions(precision=2)

# plot normalized confusion matrix
plt.figure()
plot_confusion_matrix(cnf_matrix, classes=class_names, title='Normalized confusion matrix')
plt.show()
<Figure size 432x288 with 0 Axes>

All plant species except the sunflower and the rose were well classified. The sunflower and the marguerite had problems with each other, which is quite surprising, as these two plant species do not look very similar.

In the next cell all wrongly classified pictures are plotted with predicted and actual labels.

In [206]:
x,y = test_set[0]
class_name = list(test_set.class_indices)
pictrues_length = int(round((len(mis_class)/4),0))


fig,ax=plt.subplots(pictrues_length,4)
fig.set_size_inches(15,8*15)

count=0
for i in range (pictrues_length):
    for j in range (4):
        ax[i,j].imshow(x[mis_class[count]])
        ax[i,j].set_title("Predicted Flower : "+class_name[np.argmax(y[mis_class[count]])]+"\n"+"Actual Flower : "+class_name[y_pred[mis_class[count]]])
        plt.tight_layout()
        count+=1

It seems that many images were wrongly labelled and therefore the dataset already contains errors. Also, the sunflower and daisy problem can be seen.

Declaration of originality

I confirm that I have written this paper independently and have not used any resources other than those cited. The parts of the work which are taken from the wording or the meaning of other works (including Internet sources) have been marked with the indication of the source.